A Graphical Query Language for Identifying Temporal Trends in Video Data

Previous work in video annotation and analysis has concentrated more on the process of annotation than analysis. The focus of our research instead is to provide support for video analysis by developing an environment for the “casual user” to query the video data using spatio-temporal characteristics and to review visual results for trend analysis. In this paper, we present our approach for identifying trends in video data via examining relationships between video annotations. This approach allows users to characterize the video in terms of relationships between events (e.g., events of type B frequently follow events of type A). The focus of this paper is on temporal relationships and trends. We present a graphical query language for specifying relative temporal queries between sets of annotations. This query language builds on the notion of dynamic query filters and significantly extends them. It can be used in visual information seeking to discover temporal trends in the video data.